Using implementation science to operate as a learning health system to improve outcomes in early psychosis.

AIM: Early interventions are well understood to improve psychosis outcomes, but their successful implementation remains limited. This article introduces a three-step roadmap for advancing the implementation of evidence-based practices to operate as a learning health system, which can be applied to early interventions for psychosis and is intended for an audience that is relatively new to systematic approaches to implementation. METHODS: The roadmap is grounded in implementation science, which specializes in methods to promote routine use of evidence-based innovations. The roadmap draws on learning health system principles that call for commitment of leadership, application of evidence, examination of care experiences, and study of health outcomes. Examples are discussed for each roadmap step, emphasizing both data- and stakeholder-related considerations applicable throughout the roadmap. CONCLUSIONS: Early psychosis care is a promising topic through which to discuss the critical need to move evidence into practice. Despite remarkable advances in early psychosis interventions, population-level impact of those interventions is yet to be realized. By providing an introduction to how implementation science principles can be operationalized in a learning health system and sharing examples from early psychosis care, this article prompts inclusion of a wider audience in essential discourse on the role that implementation science can play for moving evidence into practice for other realms of psychiatric care as well. To this end, the proposed roadmap can serve as a conceptual guiding template and framework through which various psychiatric services can methodically pursue timely implementation of evidence-based interventions for higher quality care and improved outcomes.

Performing an Organizational Health Literacy Assessment in a Shelter Serving People with Mental Illness.

BACKGROUND: Health literacy research and practice are constantly evolving. Recent inquiry has highlighted the burdensome literacy demands faced by people with serious mental illness (SMI). Systems, organizational operations, and structures can play a role in decreasing literacy demand, thereby reducing unnecessary challenges for people with SMI. Brief Description of Activity: An organizational health literacy assessment was conducted to explore literacy demands in one mental health shelter and revealed best practice action steps for improving the literacy environment. IMPLEMENTATION: The assessment included an exploration of the shelter environment using The Health Literacy Environment Activity Packet, First Impressions & Walking Interview, and a commonly used shelter document using the Simple Measure of Gobbledygook (SMOG), Suitability Assessment of Materials (SAM), and Centers for Disease Control and Prevention (CDC) Index. RESULTS: The literacy demands of the shelter environment and a frequently used document exceeded the literacy skills of people with SMI. Environment assessment revealed environmental facilitators (e.g., welcoming atmosphere) and barriers (e.g., unclear signage). Document assessment also revealed facilitating factors and barriers. SMOG scores ranged from 11.25 to 11.80 (median: 11.38), meaning 11th to 12th grade-level reading skills are required to understand, use, and take action on the document's content. A SAM score of 50% (adequate) and a CDC Index score of 42.1 (revise and improve) reveal materials contain both facilitating factors (e.g., chunked sections) and barriers (e.g., jargon, mismatched graphics) to use. LESSONS LEARNED: The mismatch between system demands and the literacy skills of people with SMI is more profound than that of the general United States population. Organizational health literacy assessments are achievable and useful for both immediate and long-term action aimed at understanding and improving the organizational health literacy of mental health spaces. Further work is needed to explore the role of behavioral health services in addressing the institutional and programmatic literacy demands that inhibit treatment and recovery. [HLRP: Health Literacy Research and Practice. 2022;6(3):e167-e174.] Plain Language Summary: An organizational health literacy assessment reveals how system demands can be changed to better serve intended users. Engaging in mental health, recovery, and treatment services requires complex literacy skills. Generally, the U.S. adult population does not have the skills to meet such demands, and this is especially true among public mental health service users.

"Real-world" first-episode psychosis care in Massachusetts: Lessons learned from a pilot implementation of harmonized data collection.

AIM: Increasing evidence points to the value of coordinated specialty care (CSC) for early intervention in psychotic disorders. This report characterizes clinical and socio-demographic features of patients at CSC programs in Massachusetts (MA), assessed by a standardized battery incorporated into "real-world" clinical care. METHODS: The MA psychosis network for early treatment developed a pilot battery to coordinate assessments across six CSC clinics. Programs reported baseline, 6-month, and 12-month data from a sample of 287 patients with intake dates ranging from April 2015 to December 2020. RESULTS: Patients showed improvements in functioning, emergency service use and several symptom domains at 6 and 12 months. Missing data proved to be a limitation. CONCLUSIONS: Patients improved on several meaningful domains within the first year of CSC treatment. Future implementation efforts in cross-program data collection should consider strategies to circumvent limitations related to heterogeneity between clinics, patient discharge and clinics' capacity for data collection.

Changes in community providers' screening behaviours, referral practices, and clinical confidence following participation in an early psychosis educational campaign.

AIM: Successful delivery of care to individuals with early psychosis depends on the ability of community providers to identify and refer appropriate candidates for services. Although specialty centres commonly rely upon education and outreach campaigns to building bridges with community providers, few studies have examined the effectiveness of these campaigns or the mechanisms by which they may achieve their intended effects. METHODS: We surveyed community clinicians (N = 39) about their screening behaviours, referral practices, and confidence in managing early psychosis just before and 3-6 months after attending an educational event designed to promote recognition and quality treatment of early psychosis. RESULTS: Three to six months following attendance, providers reported screening a greater proportion of clients for early psychosis, referring a greater number of clients to specialty services, and feeling more confident in their ability to respond to clients with early psychosis. Increases in confidence following attendance were associated with corresponding increases in screening behaviour. CONCLUSIONS: The results suggest that outreach campaigns designed to enhance community providers' knowledge about early psychosis assessment and resources may be effective in promoting screening, referrals, and confidence in managing psychosis. Gains in provider confidence may contribute to increases in screening. Given the lack of control group and relatively short follow-up period, more research is needed to determine the effects of early psychosis educational events and the mechanisms by which they may promote successful treatment delivery for young people in need.

Prevalence and Predictors of Limited Literacy in Public Mental Health Care.

Literacy is an important predictor of health care utilization and outcomes. We examine literacy among people seeking care in a state funded mental health clinic (Site 1) and a safety-net hospital clinic (Site 2). Limited literacy was defined as literacy at or below the 8th grade level. At Site 1, 53% of participants had limited reading literacy and 78% had limited aural literacy. At Site 2, 72% had limited reading and 90% had limited aural literacy. Regression analyses examined associations among limited literacy and psychiatric, neurocognitive and sociodemographic characteristics. Few associations among psychiatric and neurocognitive factors, and literacy were found. At Site 2, black and "other" race participants had higher odds for limited literacy compared to white participants suggesting that limited literacy may be an under-examined mechanism in understanding racial disparities in mental health. Work is needed to understand the relationships among literacy, mental health and mental health care.

What matters: Factors impacting the recovery process among outpatient mental health service users.

OBJECTIVE: Understanding factors that promote and hinder the recovery process for people living with serious mental illness remains of critical importance. We examine factors, including limited literacy, associated with mental health recovery among public mental health service users. METHOD: This study uses data from a mixed-methods, service-user informed project focused on the impact of limited literacy in the lives of people with serious mental illness. Data from structured interviews evaluate perceptions of recovery as assessed with the Recovery Assessment Scale (RAS). Regression models examine factors related to recovery controlling for sociodemographic factors, literacy, neurocognition, mental health status, perceived social support, and stigma. RESULTS: Despite bivariate relationships between RAS and limited literacy, the full models suggest that other factors account for this relationship. These include mental health status, higher social support, higher self-reported community status, and higher stigma consciousness, as well as race for some models. CONCLUSIONS AND IMPLICATIONS FOR PRACTICE: Our findings that social support and perceptions of community status are associated with higher scores on the RAS echo prior work demonstrating the importance of social connection and context in mental health recovery. Though literacy was not a predictor of recovery, further research should examine the relationship between literacy and recovery given the deep literature on literacy on health outcomes. In order to better support people in the recovery process it is important that more research is done to examine the complex relationship between stigma consciousness and recovery as well as understand the racial disparities that exist within the recovery subscales. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Implementation case study: Multifamily group intervention in first-episode psychosis programs.

AIM: Family interventions are a core component of first-episode psychosis (FEP) treatment; however, low implementation rates are consistently reported. As such, work is needed to understand the factors impacting real-world treatment delivery. The present paper describes the implementation of the McFarlane-model multifamily psychoeducational groups (MFG) in established FEP early intervention programs within a single state. The aims were to examine (a) training participation and implementation of MFG, (b) barriers and facilitators to implementation, and (c) modifications made to MFG. METHODS: Practitioners from six established FEP early intervention programs received in-person training and ongoing consultation in MFG. Training participation data were obtained via attendance and implementation outcomes were obtained from practitioner reports. Fifteen months following the initial training, practitioners reported on clinic-specific barriers, facilitators, and modifications across four categories (context, intervention, practitioner, and recipient). RESULTS: Twenty-three practitioners across six clinics received in-person training and were offered ongoing consultation to support implementation. Difficulties in starting MFG were salient as the earliest group was run 7 months after the initial training, thereby resulting in low overall frequency of groups. A number of barriers spanning context, intervention, practitioner, and recipient domains were noted, the majority of which were clinic-specific. Despite challenges, practitioners identified several facilitators and made modifications to the intervention and its delivery in service of implementation. CONCLUSIONS: Results from this implementation case study highlighted the challenges of delivering MFG in real-world FEP early intervention programs. Further, this paper emphasizes the value in identifying and addressing clinic-specific factors when implementing MFG.

Development of a Boston Treatment Program for Youth at Clinical High Risk for Psychosis: Center for Early Detection, Assessment, and Response to Risk (CEDAR).

Over the past two decades, increasing attention has been given to the importance of early intervention for psychosis. This article describes the development of the Center for Early Detection, Assessment and Response to Risk (CEDAR), which focuses on early identification and treatment of youth at clinical high risk for psychosis. There are relatively few models in the United States for such programs, and we present our developmental story, focusing mainly on the CEDAR Clinic, as a case study of how such a program can develop. We describe the rationale, infrastructure, and services provided at the CEDAR Clinic, and present some descriptive data from the CEDAR Clinic through 2016. A case example is provided to illustrate treatment at CEDAR. We hope that the cultural history of our program's development is informative for clinicians and policy makers as one model of how to build an early intervention service. We believe that this article is timely in view of the growing momentum in the United States for developing programs for intervening as early as possible for youth at clinical high risk for psychosis.

Development and validation of the client engagement and service use scale: A pilot study.

Specialized treatment for first episode psychosis offers clients a menu of services coordinated within a specialized treatment team. To enhance the impact of these services, promoting engagement and preventing early treatment drop-out is critical. However, engagement is poorly tracked and typically quantified through proxy variables such as session attendance, medication adherence, or working alliance. The aim of this study is to introduce and pilot a new measure of engagement for first episode psychosis coordinated specialty care, the Client Engagement and Service Use Scale (CENSUS). The CENSUS was evaluated for reliability and validated against the Service Engagement Scale and an appointment count for a small sample (N = 10) of first episode clients. The measure was also evaluated for acceptability by a consumer advocacy group. Clinicians achieved high inter-rater reliability after minimal training. CENSUS items demonstrated medium to large correlations with other measures of engagement. Feedback from the consumer group emphasized that clinicians should ask questions in a way that is nonjudgmental and successfully elicits authentic client feedback about their service preferences. This pilot study yielded preliminary evidence of reliability and validity, suggesting that the CENSUS is a useful and novel tool for tracking and differentiating degrees of client engagement across multiple intervention components and for facilitating structured discussions regarding clients' service utilization and preferences.

Psychosocial approaches to violence and aggression: contextually anchored and trauma-informed interventions.

Psychosocial interventions are part of the complex understanding and treatment of violent behavior in our state mental health hospitals. A comprehensive assessment of violence and aggression includes attention to all 3 domains of prevention and assessment (primary-institutional, secondary-structural, and tertiary-direct). Trauma experiences and their consequences may include behavioral violence and aggression. The authors' premise is that trauma is a universal component in the individual assessment of violent behavior. Therapeutic interventions must include a trauma-informed formulation to be effective. Organizational commitment to trauma-informed, person-centered, recovery-oriented (TPR) care is crucial to the efficacy of any of the interventions discussed. Thus, the dynamic nature of the individual, interpersonal, environmental, and cultural factors associated with the daily operations of the inpatient unit need to be assessed through the lens of primary and secondary violence prevention, building on the recognition that the majority of persons served and staff have significant trauma histories. Once a compassionate, respectful, empathic, and empowering approach is embraced by leadership and staff, the work with individuals can proceed more effectively. Interventions used include a variety of cognitive-behavioral, interpersonal, and somatosensory therapies. These interventions, when effectively applied, result in more self-esteem, self-mastery, self-control for the person served, and diminished behavioral violence.

The impact of premorbid adjustment, neurocognition, and depression on social and role functioning in patients in an early psychosis treatment program.

OBJECTIVE: Functional impairments are debilitating concomitants of psychotic disorders and are present early in the illness course and, commonly, prior to psychosis onset. The factors affecting social and role functioning in early psychosis (EP) following treatment are unclear. We evaluated whether six months of participation in the PREP(R), Boston, EP treatment program, part of a public-academic community mental health center, was related to improvements in social and role functioning and whether premorbid adjustment in adolescence, baseline neurocognition, and depression symptoms predicted functional improvement. METHOD: The Global Functioning Social and Role scales, MATRICS neurocognitive battery, and Calgary Depression Scale were assessed at baseline and six months during naturalistic treatment, while premorbid adjustment was measured at baseline. All participants were psychotic disorder patients in PREP(R) (n = 46 with social functioning and 47 with role functioning measures at both time points). RESULTS: Large improvements were observed in role functioning (d = 0.84) and medium to large improvements were observed in social functioning (d = 0.70). Models consisting of adolescent premorbid adjustment and change in depression symptoms predicted social and role functioning change, whereas neuropsychological functioning did not. CONCLUSIONS: Substantial improvements in social and role functioning were observed among this sample participating in a recovery-based EP program. The impact of clinical factors on social and role functioning was highlighted. Further studies of premorbid adjustment in adolescence and the treatment of depression in EP programs in controlled treatment trials are needed to confirm these findings.

Prevention and recovery in early psychosis (PREP(®)): building a public-academic partnership program in Massachusetts, United States.

Recently, there has been increasing emphasis on early intervention (EI) for psychotic disorders. EI programs in public mental health settings have been established in countries such as Australia, the United Kingdom, and Canada. However, there are relatively few EI programs in the United States (U.S.). Here we describe the conceptual origins and practical development of the PREP program, i.e., Prevention and Recovery in Early Psychosis, as it evolved in a public academic psychiatry setting in Boston, Massachusetts, U.S. PREP developed over a decade through a partnership between the Massachusetts Department of Mental Health and academic institutions within the Harvard Department of Psychiatry. We discuss the evolution, programmatic features, funding mechanisms, staffing, and the role of clinical training in PREP. The key principles in developing the PREP Program include the focus on early, evidence based, person-centered and phase-specific, integrated and continuous, comprehensive care. This program has served as a foundation for the emergence of related services at our institution, including a research clinic treating those at clinical high risk or within the putative "prodromal" period preceding frank psychosis. This account offers one possible blueprint for the development of EI programs despite the lack in the U.S. of a national mandate for EI or prevention-based mental health programs.

National comorbidity survey replication adolescent supplement (NCS-A): III. Concordance of DSM-IV/CIDI diagnoses with clinical reassessments.

OBJECTIVE: To report results of the clinical reappraisal study of lifetime DSM-IV diagnoses based on the fully structured lay-administered World Health Organization Composite International Diagnostic Interview (CIDI) Version 3.0 in the U.S. National Comorbidity Survey Replication Adolescent Supplement (NCS-A). METHOD: Blinded clinical reappraisal interviews with a probability subsample of 347 NCS-A respondents were administered using the Schedule for Affective Disorders and Schizophrenia for School-Age Children (K-SADS) as the gold standard. The DSM-IV/CIDI cases were oversampled, and the clinical reappraisal sample was weighted to adjust for this oversampling. RESULTS: Good aggregate consistency was found between CIDI and K-SADS prevalence estimates, although CIDI estimates were meaningfully higher than K-SADS estimates for specific phobia (51.2%) and oppositional defiant disorder (38.7%). Estimated prevalence of any disorder, in comparison, was only slightly higher in the CIDI than K-SADS (8.3%). Strong individual-level CIDI versus K-SADS concordance was found for most diagnoses. Area under the receiver operating characteristic curve, a measure of classification accuracy not influenced by prevalence, was 0.88 for any anxiety disorder, 0.89 for any mood disorder, 0.84 for any disruptive behavior disorder, 0.94 for any substance disorder, and 0.87 for any disorder. Although area under the receiver operating characteristic curve was unacceptably low for alcohol dependence and bipolar I and II disorders, these problems were resolved by aggregation with alcohol abuse and bipolar I disorder, respectively. Logistic regression analysis documented that consideration of CIDI symptom-level data significantly improved prediction of some K-SADS diagnoses. CONCLUSIONS: These results document that the diagnoses made in the NCS-A based on the CIDI have generally good concordance with blinded clinical diagnoses.

Validity of the assessment of bipolar spectrum disorders in the WHO CIDI 3.0.

OBJECTIVE: Although growing interest exists in the bipolar spectrum, fully structured diagnostic interviews might not accurately assess bipolar spectrum disorders. A validity study was carried out for diagnoses of threshold and sub-threshold bipolar disorders (BPD) based on the WHO Composite International Diagnostic Interview (CIDI) in the National Comorbidity Survey Replication (NCS-R). CIDI BPD screening scales were also evaluated. METHOD: The NCS-R is a nationally representative US household population survey (n=9282 using CIDI to assess DSM-IV disorders. CIDI diagnoses were evaluated in blinded clinical reappraisal interviews using the non-patient version of the Structured Clinical Interview for DSM-IV (SCID). RESULTS: Excellent CIDI-SCID concordance was found for lifetime BP-I (AUC=.99 kappa=.88, PPV=.79, NPV=1.0), either BP-II or sub-threshold BPD (AUC=.96, kappa=.88, PPV=.85, NPV=.99), and overall bipolar spectrum disorders (i.e., BP-I/II or sub-threshold BPD; AUC=.99, kappa=.94, PPV=.88, NPV=1.0). Concordance was lower for BP-II (AUC=.83, kappa=.50, PPV=.41, NPV=.99) and sub-threshold BPD (AUC=.73, kappa=.51, PPV=.58, NPV=.99). The CIDI was unbiased compared to the SCID, yielding a lifetime bipolar spectrum disorders prevalence estimate of 4.4%. Brief CIDI-based screening scales detected 67-96% of true cases with positive predictive value of 31-52%. LIMITATION: CIDI prevalence estimates are still probably conservative, though, but might be improved with future CIDI revisions based on new methodological studies with a clinical assessment more sensitive than the SCID to sub-threshold BPD. CONCLUSIONS: Bipolar spectrum disorders are much more prevalent than previously realized. The CIDI is capable of generating conservative diagnoses of both threshold and sub-threshold BPD. Short CIDI-based scales are useful screens for BPD.

Concordance of the Composite International Diagnostic Interview Version 3.0 (CIDI 3.0) with standardized clinical assessments in the WHO World Mental Health surveys.

The DSM-IV diagnoses generated by the fully structured lay-administered Composite International Diagnostic Interview Version 3.0 (CIDI 3.0) in the WHO World Mental Health (WMH) surveys were compared to diagnoses based on follow-up interviews with the clinician-administered non-patient edition of the Structured Clinical Interview for DSM-IV (SCID) in probability subsamples of the WMH surveys in France, Italy, Spain, and the US. CIDI cases were oversampled. The clinical reappraisal samples were weighted to adjust for this oversampling. Separate samples were assessed for lifetime and 12-month prevalence. Moderate to good individual-level CIDI-SCID concordance was found for lifetime prevalence estimates of most disorders. The area under the ROC curve (AUC, a measure of classification accuracy that is not influenced by disorder prevalence) was 0.76 for the dichotomous classification of having any of the lifetime DSM-IV anxiety, mood and substance disorders assessed in the surveys and in the range 0.62-0.93 for individual disorders, with an inter-quartile range (IQR) of 0.71-0.86. Concordance increased when CIDI symptom-level data were added to predict SCID diagnoses in logistic regression equations. AUC for individual disorders in these equations was in the range 0.74-0.99, with an IQR of 0.87-0.96. CIDI lifetime prevalence estimates were generally conservative relative to SCID estimates. CIDI-SCID concordance for 12-month prevalence estimates could be studied powerfully only for two disorder classes, any anxiety disorder (AUC = 0.88) and any mood disorder (AUC = 0.83). As with lifetime prevalence, 12-month concordance improved when CIDI symptom-level data were added to predict SCID diagnoses. CIDI 12-month prevalence estimates were unbiased relative to SCID estimates. The validity of the CIDI is likely to be under-estimated in these comparisons due to the fact that the reliability of the SCID diagnoses, which is presumably less than perfect, sets a ceiling on maximum CIDI-SCID concordance.

The prevalence and correlates of nonaffective psychosis in the National Comorbidity Survey Replication (NCS-R).

BACKGROUND: To estimate the prevalence and correlates of clinician-diagnosed DSM-IV nonaffective psychosis (NAP) in a national household survey. METHODS: Data came from the United States National Comorbidity Survey Replication (NCS-R). A screen for NAP was followed by blinded sub-sample clinical reappraisal interviews. Logistic regression was used to impute clinical diagnoses to respondents who were not re-interviewed. The method of Multiple Imputation (MI) was used to estimate prevalence and correlates. RESULTS: Clinician-diagnosed NAP was well predicted by the screen (area under the curve [AUC] = .80). The MI prevalence estimate of NAP (standard error in parentheses) is 5.0 (2.6) per 1000 population lifetime and 3.0 (2.2) per 1000 past 12 months. The vast majority (79.4%) of lifetime and 12-month (63.7%) cases met criteria for other DSM-IV hierarchy-free disorders. Fifty-eight percent of 12-month cases were in treatment, most in the mental health specialty sector. CONCLUSIONS: The screen for NAP in the NCS-R greatly improved on previous epidemiological surveys in reducing false positives, but coding of open-ended screening scale responses was still needed to achieve accurate prediction. The lower prevalence estimate than in total-population incidence studies raises concerns that systematic nonresponse bias causes downward bias in survey prevalence estimates of NAP.

Clinical calibration of DSM-IV diagnoses in the World Mental Health (WMH) version of the World Health Organization (WHO) Composite International Diagnostic Interview (WMHCIDI).

An overview is presented of the rationale, design, and analysis plan for the WMH-CIDI clinical calibration studies. As no clinical gold standard assessment is available for the DSM-IV disorders assessed in the WMH-CIDI, we adopted the goal of calibration rather than validation; that is, we asked whether WMH-CIDI diagnoses are 'consistent' with diagnoses based on a state-of-the-art clinical research diagnostic interview (SCID; Structured Clinical Interview for DSM-IV) rather than whether they are 'correct'. Consistency is evaluated both at the aggregate level (consistency of WMH-CIDI and SCID prevalence estimates) and at the individual level (consistency of WMH-CIDI and SCID diagnostic classifications). Although conventional statistics (sensitivity, specificity, Cohen's kappa) are used to describe diagnostic consistency, an argument is made for considering the area under the receiver operator curve (AUC) to be a more useful general-purpose measure of consistency. In addition, more detailed analyses are used to evaluate consistency on a substantive level. These analyses begin by estimating prediction equations in a clinical calibration subsample, with WMH-CIDI symptom-level data used to predict SCID diagnoses, and using the coefficients from these equations to assign predicted probabilities of SCID diagnoses to each respondent in the remainder of the sample. Substantive analyses then investigate whether estimates of prevalence and associations when based on WMH-CIDI diagnoses are consistent with those based on predicted SCID diagnoses. Multiple imputation is used to adjust estimated standard errors for the imprecision introduced by SCID diagnoses being imputed under a model rather than measured directly. A brief illustration of this approach is presented in comparing the precision of SCID and predicted SCID estimates of prevalence and correlates under varying sample designs.